A Node Based Filter Editor For Inkscape:
This Document is also available as an web page, with formatting the way it was intended, at https://phantomzback.github.io/GsocProposal24/ . I've authored the document as a markdown file in Obsidian, with a theme applied which I feel is a great look for documents. However, exporting this to a PDF causes significant loss of formatting and a general loss in appeal. I've done my level best to try to format the document to the best of my abilities but it still has hiccups. Therefore, I would request you to instead use the web page to read through my proposal. My humblest apologies for the inconvenience
This proposal aims to be a detailed document, for my proposal for a Node Based Filter Editor for Inkscape. It contains the summary of all my research, discoveries, learnings and progress with this, and the conclusions I've drawn from them.
I hope this acts as a testimony of my abilities, and I hope I am able to convince you that I am up for this task! So here goes...
- When did you first hear about inkscape?
Inkscape and I go wayyy back. I started off programming with Scratch in 2014, when I was in class 4. I once came across Vector and Bitmap somewhere on scratch and asked my father to explain the difference, and I was amazed! Naturally, I wanted to create them. So I found out about Inkscape and started tinkering right away. It served as my starting point to getting to know about the world of Open-Source. I was hooked to this concept and found out about a lot more, GIMP, Blender, Krita, Synfig, games like 0 A.D., .... WHAT NOT! and switched my workflow to be completely open source. (All the diagrams attached here, as well as the document are made in open source software :P).
- What kind of drawings do you create with Inkscape?
I started out making drawings from tutorials and very basic drawings on my own, since I was young and new to the concept. Eventually, I got around to creating logos and arts inspired by the channel Logos By Nick, since they totally mesmerised me! Then, I began creating flat sprites to create animations like Kurzgesagt , and animating them using blender. (Then I tried making my own software for Vector Animations )
- Describe your participation in our community (e.g. uploaded drawings, tutorials, bug reports, communication via mailing lists or IRC).
I was an active member on Inkscape discord channel, until my entrance exam preparations started in 2021. Unfortunately, I remained unaware of the IRC and RocketChat channels until recently, and thus didn't communicate on the platform. But now I am an active member on the development and general channels on Inkscape's RocketChat. Since I've been an active user of Inkscape since long, I've also provided help on the Inkscape discord, and now on the RocketChat.
- Describe your contributions to the Inkscape development (e.g. bug fixes, translations, packaging, testing).
I have published bug fixes (this and this). I am also currently working on improvements in several areas on my local branches.
A bit of an additional background: I first had a look and cloned Inkscape's source code in 2019 (I was just 14!), since I wanted to create my own SVG animation software, and I was overwhelmed looking at the source code (just like many of the developers shared in Martin Owen's videos, no doubt the source code is large). After trying for a day to try to separate the XML and Renderer from the Inkscape Source Code to use in mine, I gave up and felt it was too tough for me. Now, after having completed my college entrance exams, I have gotten around to being able to not only understand the source code, but develop patches, improvements, and hopefully, a TOTALLY NEW FEATURE!
- In exactly two sentences, why should we pick YOU?
- What is your name, email address, and IRC nickname?
Name: Ravi Arora
Personal Email Address: yugiarora@gmail.com
College Email Address: raviarora23@iitk.ac.in
RocketChat Nickname: Ravi.Arora (aka, GarbCol05.aka.PhantomzBack, aka, GarbCol05)
Please Note: I initially joined with some aliases which caused confusion on the chat, apologies for that!
General Alias: PhantomzBack (github, gitlab, discord, etc.)
- Describe any plans you have for the summer in addition to GSoC (classes, thesis, job, vacation, etc.).
I would be travelling for 2-3 weeks (overlaps with the community bonding period) visiting my hometown, but can continue to work from there. Apart from that, I don't have any plans.
I can commit to a minimum of 30 hours and ideally, 40-50 hours in week. I would be available at most of the times on Inkscape's RocketChat.
I am currently located in India (UTC+5:30), and am usually available from 12 P.M. to 4 A.M. IST
- What programming projects have you completed?
I have worked on an innumerable number of programming projects, the most prestigious one being a Qt and CLI-based SVG Animation Software, which I designed from scratch.
It started off with me being annoyed that I couldn't find any alternative for Adobe After Effects, after discovering Kurzgesagt used AE for creating their animations. I wanted to give Adobe open source competition for this too (eventually turning my software into a layer based compositor), like Inkscape and GIMP did for Illustrator and Photoshop!
The GUI was not the best since I had to leave the project midway to focus on college entrance examinations, but it provided ability to animate elements inside SVGs, and interpolate properties.
Here's a broader collection of my projects
pygamepygamePyQt5 based printer interface appopencv, matplotlib, in Jupyter NotebooksManim - 3Blue1Brown's mathematical-animation libraryBeautifulSoup) and tKinterdiscord.pyHTML, CSS, php sites for learning (LAMP stack)ReactJS web apps for learning.Node.js tool to export HTML to images using headless chromium
- What are your favourite programming tools (editor, etc.)?
Briefly,
Reasons:
I love using VSCodium - open source VS Code bindings, due to innumerable number of extensions, the clean looks and the ease of usage. In past, I did use Emacs but didn't feel that was the perfect fit for me. I love the fact that I have features like SVG viewers, live servers, etc. The flexibility in navigating code, peeking definitions, references, etc. make it wonderful. Extensions such as Anchors, Bookmarks make it even easier to store important references and tags.
I love to make use of Obsidian for making notes wherever I need to (this document is written in Obsidian too) . But, I am considering switching to Logseq. I also keep my tab S9 handy with me to draw diagrams to think and plan out while programming.
Writing python code is easy to write and quickly written, very suitable for simple tasks.
- Describe any work on other open-source projects.
- List other GSoC projects you are applying to.
None, apart from this one on Inkscape :).
#ux
The current filter editor, is a layer based filter editor, with various drawbacks, and is not very user friendly. It is not very intuitive to use, and doesn't provide great user-experience, severely slowing them down.
The filter editor, is highly capable of creating some wonderful effects, but the UI is extremely non-intuitive for beginners, and drives them away. The current editor makes it really hard to understand what's going on.
For starters, the ease to work with decreases significantly as we increase the number of primitive effects in the filter. It is fairly annoying to understand the connections in the current format and the connections get seemingly more and more frustrating. Here's an example of what the editor looks like after applying just two filters from the Filter Gallery.
Users are limited to viewing parameters of only one primitive filter at a time. This creates an unsatisfactory user experience, since quite often, users would wish for ease in modifying parameters of multiple filters at a time, to get the desired look they wish to achieve. Doing so right now would be a tedious task, but this would be wonderfully resolved with the help of a Node-based workflow.
Making connections between various filters, or the source channels, has no highlighting, and is not tactile.
There's no simple way to preview the effects of a filter up-to some primitive filters. There are workarounds for this, such as using a placeholder merge node just and plug-in what we wish to see into it, but such methods are undesirable. However, this can be easily integrated in a node based editor by giving the option of providing a preview up-to each filter.
The primitive filter editor gets very cluttered, which can easily be avoided with node groups, similar to the functionality offered in blender.
The plan has been broken up into various segments, I have also added created tags for the reader/reviewer to be able to easily explore parts relevant to them, feel free to make use of them. Although, it is recommended to go through all of it to understand the complete flow and plan, with no stone left unturned, and since their is considerable overlap of topics at various places. This segment also consists of my research and conclusions.
The tags for navigation are:
#technical - Targeting development
#ux - Targeting UI/UX discussions
#important - Do go through these.
#important #technical #ux
I have put in a lot of hours into refining and doing my best to perfect my proposal and application for this project, and giving a clear and detailed analysis to reflect on my abilities to take on such a task. To accomplish this, I made it my goal to approach this in a systematic manner and with a clear vision, and start the work by building a clear foundation, and not hastily rushing into getting started with the development, to also support future prospects by creating re-usable functionality by making abstracted and templated classes.
#ux To get started, I put in great efforts into analysing the existing editor, finding out flaws in the current one, taking reviews on problems faced by users in the current one, figuring out whether a node editor would really be the best replacement for a filter editor, and if so, what should be the design? I analysed various existing node editor ideas to try and come up with a strategy fit for our use case (listed below in UI/UX analysis). During this time, I also focused greatly on connecting with the community, users as well as developers to try to match the overall goal. I created mock-ups after discussing with various members on the group, I researched various existing node editors to come up with ideas for how to design it to be a good fit in Inkscape. I had constant healthy discussions with various members, took polls and reviews on Reddit and the Inkscape discord, and Rocket Chat, about features lacking in the current one, and what they really needed, and spent time modelling the designs and concepts.
#technical Rather than putting my focus on creating a prototype, I shifted my focus on figuring out the Best Strategy for implementation. I researched on libraries which offer functionality like a Node-based editor, libgtkflow and gtknodes, and each of these had their fair set of pro's and con's. A detailed analysis is given in the appendix (libgtkflow vs gtknodes). I concluded that it is a better decision to not stick to either of these. I also tried to find various GTK based software to try to figure out if any have a node based editor which we could possibly extract, or take inspiration from, but unfortunately, couldn't find any. So, I concluded that it's best to create a Node based Editor from scratch, taking inspiration from them. Each part of it tailored to fit Inkscape, abstracted and implemented the way we want it, and set up a basis for future similar editors in the future. And it was also necessary to see whether going along the GTK Widgets approach, rather than a custom drawing area would be a better approach. For example, in my research, I understood that zoom is definitely not an intuitive thing to tackle when working with Gtk Layouts, Windows, etc.. And this would be a deal breaker, since everyone's display has varying DPI, it's highly highly inconvenient to constantly keep resizing widgets or be forced to one view.
After doing my clear ground research, I implemented a prototype. It is discussed ahead.
Here are the mock-ups I have designed while discussing with development team, as well as UX team. I am still working on coming up with more, and would be showcasing them on the relevant channels soon! I've been a bit mischevious with the themes and color choices, and tried coming up with fresh ideas rather than solely matching Inkscape's design, but this was just for me to try out. The final mock-ups would fit in well with Inkscape's theme
These are designed after taking inspiration from various Node based editors (Blender's Material Editor, Nodebox's Node Based Editor, Enso's node based editor)
A More Complex Node:
Pros:
Cons:
Pros:
Cons:
#future An Illustration for how Preview can be incorporated:
A fold-able panel for each node, allowing the user to toggle it for each node.
This allows the user to be able to preview the output upto multiple filters simultaneously, and also allow the user to see the effect applied by an individual filter.
A prototype meant to showcase what you can expect and prove my ability to be able to take on this task. A video showcasing the various elements of the prototype is available here (contains audio cues)
An image of the Prototype in action:
Works With Zoom:
The prototype features:
Additionally, the behaviour also works with zoom, but the code for that was incomplete and hence transferred to another branch.
Functional Node Based Filter Effects Editor with the following Features:
From the UI/UX perspective: #ux
Functional Node based Filter Editor replacing all the existing Filter Editor functionality, in a well thought, intuitive, user friendly, with some of the key features higlighted below:
Node Editor Canvas:
Filters System And Integration With Document:
Preserving the Node Editors State:
This lists out some implementations I can add to the editor, if time permits, although some of these might be overly ambitious for now! The tag #future highlights these at other places.
libgtkflow and the code there can be used for reference)May 1 - May 26:
Community Bonding Period:
lib2geom classes, and how they can be used to handle geometrical options of the canvas to simplify geometry management. This also serves to ensure uniformity in source code, rather than coming up with new classes for geometry management for the canvas.testfiles and resources on Testing to get familiar with how to create effective tests, get an idea of how to develop a Unit Test. May 27:
Coding Period Begins:
I've condensed the completion of tasks into a period where I can implement them comfortably, but often, things go wrong and take unexpected turns. In light of this, I've kept time in the last few weeks as buffer to catch up in case of such events, and kept light tasks towards the end. Needless to say, I would always try to be running ahead of the clock! Also, some of these are bound to change to some extent based on conclusions drawn from further discussions. Additionally, the GTK4 branch's filter editor is currently broken. Until it is recovered to a stable state (I will try contributing in this), I may work on the 1.4.x branch and forward port the code later.
#prototype are features I have already showcased in my prototype
#research are features/tasks which would require a significant amount of time for me to think and research before being able to tackle (please don't get me wrong, I would be requiring research and thinking for every task, but tasks tagged with this would require a major amount of research first, because I have made little to no progress in understanding these and planning how I would implement these, or lack any idea of implementing this at all)
Please Note: Mostly, if there are no changes in the technical specifications and implementation method, implementing the canvas can begin from within Inkscape, due to the nature of my plan, since these widgets can safely replace the widgets in the current filter editors dialog, making the integration into Inkscape easier. Beginning from within Inkscape is also probably a better approach, so that event handling (context menu, clicks, modifiers, hotkeys, etc.) can be inherited the way Inkscape does it, rather than creating controllers for event handling in a dummy app, and then putting in more efforts to port it into Inkscape. So, I would like to start my development from within Inkscape!
Week 1: May 27 to June 2, Week 2: June 3 to June 9
FilterEffectsDialog to have a placebo node editor in placeThis would handle the integration in terms of bringing the widgets to Inkscape. The integration of connecting these widgets to filters in a functional manner is explained ahead.
Week 3: June 10 to June 16, Week 4: June 17 to June 23
SpinButtonAttr, MatrixAttr, EntryAttr, FileOrElementChooser, etc.Week 5: June 24 to June 30, extends into Week 6
AttrWidget) to the corresponding primitives (if a separate parameter dialog is not used, else, mapping the layouts onto the parameter handling widget)SPFilter slotsWeek 6: July 1 to July 7
Midterm Evaluation: July 8 to July 12
Week 7: July 8 to July 14, extends into week 8
Week 8: July 15 to July 21
DocumentUndo::done suffice? Need to understand this functionality a lot better)Week 9: July 22 to July 28, Week 10: July 29 to August 4
Week 11: August 5 to August 11, Week 12: August 12 to August 18
Final Week: August 19 to August 26:
There are two broad approaches
#technical
DrawingArea or GLArea
DrawingArea and GLArea do not directly supporting placing widgets on top of them. They are basically plain widgets which give us easier drawing API's, the same effect can be reproduced on any other widgets by overriding snapshot-vfunc in GTK4 and OnDrawSignal in GTK3.Firstly, reusing the code!
A lot of behaviour we are looking for has already been implemented, such as,
lib2geom (to be used in the same way it is for the canvas )
canvas_grid.cpp - contains the logic for the scrollbars, adjustments, as well as their signals (specifically). The deskarea, _canvas->get_area_world(), viewbox use Geom::Rect to calculate the bounds of the allocated canvas. The same can be done for the widgets canvas, except using recursive Gtk::Viewport and Gtk::Fixed's to handle this. The complete approach is attached in the canvas section below.SPFilterPrimitive and SPFilter are used, how the editor is setup to reflect changes in the editor with changes in the document, how the current UI is wired, etc. This provides great insight in the approach to use while replacing the filter editor. After researching and testing out the available classes from GTK, I came up with two approaches for implementing an infinite canvas with support for panning, zooming, etc.
Approach 1: Based on Scrolled Window
The prototype is based on this approach.
Rough Class Structure:
canvas_sw : Gtk::ScrolledWindow
canvas_viewport : Gtk::Viewport
canvas_outer : Gtk::Fixed
canvas_inner : Gtk::FixedViewport gives us adjustments which we can as well as the ability to place our child widgets as we wish (this could be replaced with a custom layout manager too, or a Gtk::Fixed to place outer and manage it's placement, which would require us to re-implement some functionality, such as the adjustments, and handling their update signals)
We use inner for having the ability to place our widgets wherever we wish inside the canvas area. This gives us control over being able to place our nodes freely.
outer -> this may seem unnecessary, but outer is crucial for implementing zoom, since implementing zoom is probably one of the hardest and non-intuitive parts of this, since GTK offers no native support from zoom. I plan on implementing zoom using CSS transform: scale(). However, due to a prevailing inconsistency in GTK's source code, transform: scale(2) does not update the widgets dimensions correctly, and instead causes it to overflow. So, we use an outer window, which adjusts it's size to be able to contain the overflow, when we capture a zooom event from the user.
Drawing of connections is taken care of on Cairo contexts by overriding the snapshot-vfunc in GTK4 and on_draw in GTK3.
Event handling is taken care of by events received on canvas_inner, it is responsible for recognizing clicks, drags, etc. and taking consequent actions depending on the target
For cleaner implementation, we can override allocate and size-measure functions
The approach uses two coordinate systems, the reason should be clear below.
Schematics For Handling:
Here, viewport refers to the region visible within the bounds of the scrolled window container.
#made-in-inkscape
True Bounds is the minimum size request that inner would have.
#made-in-inkscape
Both outer and inner adjust their sizes (same when no zoom) to to accommodate the scroll.
#made-in-inkscape
The widget inner overflows from it's true allocation, we adjust the size of outer to so that the overflow can be captured in the scroll!
Setting up virtual coordinate system to handle positions of objects globally, and maintain the relation between the virtual coordinate system and the coordinate system with respect to the viewport/scrolled-window always.
Click and Drag behaviour: Implemented in the Prototype. The canvas widget maintains list of references to selected nodes, and un-selected nodes, and handles behaviours with clicks along with modifiers (Shift + Click adds to selection, Click clears selection and selects the new node, or un-selects it if it was the only node selected, etc.). Selections can also be dragged when any selected object is dragged with the cursor.
Note: This will first ask the child widget whether the click was on a Node Source/Sink and decide if a connection needs to be drawn or the selection needs to be moved.
Rubber-band Selection: Drag select on inner, find the widgets inside the selection either using Gtk provided methods (Gtk::Widget::intersect or cairo_region_intersect or manually check the widgets with bounds in this area), but a better approach would be to do the geometry management using Geom::Rect due to the ease of operators, and for uniformity in source code.
Approach 2:
Fixed and our own scrollbars to accordingly place the classes outer and inner and writing our own update functions for our scrollbars, or alternatively, writing our own widget which acts as a container for outer and inner and creating our own size-allocate and measure functions, along with scrollbars and logic for updating them.outer since it may be possible to allow viewing the overflow using the custom size-allocate functionThe final approach to use will be decided after further testing, and is subject to changes.
The rough class structure
filter_node : Gtk::Box/Gtk::Overlay
filter_node_data_container : Gtk::Box
labels and fold buttonsLayout for filter specific widgetssinks and sources What they do
filter_node: helps with overall management in placing the nodes on the canvas, and acting as a container for the rest of the node interactions, and well managed coordinate system offsetting from 0.filter_node_data_container: Parameter widgets, their respective layouts and layout managers are contained in this. This would also contain the styling for the node, if any.Filter specific widgets: These widgets are specific to the filter type. sinks and sources: Widgets which are a part of filter_node. Capturing clicks on these can convey/signal to the canvas to start creating a connection, etc. Since the click handling in the prototype is handled purely by the canvas, and the canvas further uses Gtk::Widget::pick, the canvas widgets can easily handle clicks on these and start creating connections, and store connections according to whether the drop was successful or failedCairo beziers, which should ideally to be drawn under the widgets (shown in prototype)
The reason for this is connections on top can screw up the layout, as shown in:
[Reference: libgtkflow]
Drag behaviours: #prototype
#ux : Clicking on source and then Modifier+Click, such as Ctrl+Click could be used to complete a connection rather than having to drag
#ux #future: Create sink-source style widgets, which act as sink and source, to help clean up the graph, similar to blender.
Here's the current Look in the prototypes:
Original Source, and the bottom-most node - Filter Output node. Schematic Diagram For The Logic of Auto-Layout: #made-in-inkscape
filter-effects-dialog.cpp, since it contains the code used for the current filter editorFilterEffectsDialog::add_primitive(), need not be re-written, these can be used and connected with signals set up from the Node Editor CanvasSPFilter, SPFilterPrimitive suffice in what we need, we need to only link canvas events with their respective creation, deletion, updates, etc.
SPFilter::child_addedSPFilter::remove_childSPFilter::order_changed, etc.#important #ux #technical
By Node Editor State, I refer to orientation, positions, etc. of node's, and the current view of the editor. This feature is Essential since it's important that a user is able to restore his node editor after re-opening his file, or load a filter not created in the node editor, onto it, in a readable manner.
It's essential that the node editor is able to handle:
inkscape:node:x,inkscape:node:y of the filter primitive (<fePrimitive> for the corresponding node), other attributes such as toggle visibility in inkscape:node:toggle-visibility (which is removed in the plain SVG). <filter> tag.Implementation Note: These are to be registered in the file attributes.h under SPFilter and Filter Primitive Commons
This would require registering the newly chosen attributes in the attributes.h file, then modifying the build , write, etc. methods of SPFilter and SPFilterPrimitive to incorporate the attributes we create, relevant to the state. Using these methods, we then construct the canvas, with their respective nodes and their placement, etc.
Here's all my research and planning summarised and broken down. I know it's a bit messy, I struggle with being able to organise my thoughts clearly. But, I hope in all this chaos, I have been able to prove my capability in taking on this task.
Irrespective of whether or not my proposal is accepted or not, I would love to spend my summer working on Inkscape and contributing to it's development to the best of my abilities, and learn a lot from the teams, and the members of the community!
*Hope to enjoy a wonderful summer with Inkscape and Inkscape community ahead
Signing Off,
Ravi Arora
libgtkflow vs gtknodesAn important thing to note about both these libraries is these have implementations with primary purpose (or at least, one of the primary purposes) of allowing data-exchange between nodes by setting up signals-sockets targeted for data-transmission. For majority of our use of the node-editor at present, we don't need data-transmission between nodes, as much as setting up the correct signals with the correct slots to manage connections between nodes, their is no use of transmission of any data beyond this, since the editor only sets up the order in which filter primitives are connected and their respective properties, making this feature redundant. From my understanding, we don't need to communicate this data between nodes since there's no flow of data we are trying to achieve, it's more about specifying the order in which the nodes (representing the primitives) are connected and each node should be concerned with just it's own priorities. Data flow would rather be needed for a purpose such as if each node actually processed an image, and then showed it, and then passed it on to the next one. So it might seem like data-flow may be needed for a preview node then (to be implemented in future), I don't think it would be so since we fetch the data from the rendering pipeline and the filter editor would have no role in communicating that data between nodes. But in any case, it can be implemented with ease too!
Advantages of libgtkflow:
GtkWindows and GtkWidget's rather than drawing canvases, but with poor handling.Advantages of gtknodes:
Dialog for Filter Effects is stored in filter-effects-dialog.cpp
SPFilterPrimitive and SPFilter act as the document nodes for filter and filter primitives in the document.
_filter_modifier - Contains list of filters_primitive_list - Contains the list of primitives contained in the current filter
on_draw_signal used to draw the current UI of the filter editorsnapshot_vfunc in GTK4 _settings - Responsible for handling the parameters of the UI related to the current filterfilter-chemistry.cpp contains the various methods for constructing, removing, filters, adding primitives, setting their blend modes etc.filter-enums.cpp contains the enums for filters, along with their string representations in SVGs and utility objects for their conversionsfilter-effects-chooser.cpp - handling of the widget and corresponding operations on the filter primitives for Blend Mode, Opacity, etc.nr-filter.cpp , nr-filter-primitive.cpp, etc.
Filter::render function on our own, passing our own resolution to it and intercepting the surface and getting the image data, for each filter.filter.cpp - management and methods for handling the pre-made filtersfilter-all.cpp - contains the pre-made filters available in the filters drop-down.